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EXECUTIVE  SUMMARY 


The  Army's  new  direction  in  materiel  testing  has  been  in  effect  since 
January  1975.  The  new  policy  called  Single  Integrated  Development  Test 
Cycle  (SIDTC)  was  implemented  to  provide  a more  effective  and  efficient 
materiel  testing  proccsss.  Essentially  the  objective  of  the  new  policy 
is  to  reduce  cost  and  schedule  by  eliminating  unnecessary  duplicate  testing 
which  occurred  frequently  under  the  previous  test  policy.  The  Single 
Integrated  Development  Test  Cycle  philosophy  places  increased  emphasis  on 
independent  evaluation  and  less  emphasis  on  independent  testing.  The 
change  in  test  policy  also  required  significant  changes  in  some  of  the 
traditional  roles  of  key  Army  test  participants  such  as  the  materiel 
developer,  development  tester,  and  test  evaluator.  In  the  Single  Integrated 
Development  Test  Cycle  the  role  of  contractor  testing  in  satisfying  Govern- 
ment test  requirements  is  greatly  expanded.  This  study  report  examines 
the  rationale  leading  to  implementation  of  the  Single  Integrated  Develop- 
ment Test  Cycle  policy,  the  responsibilities  of  key  participants,  and  how 
the  system  is  working.  The  information  acquired  should  be  of  interest  to 
those  involved  in  management  and  execution  of  Army  materiel  testing. 
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SECTION  I 
INTRODUCTION 

1 .1  Purpose: 

The  purpose  of  this  study  paper  is  to  acquire  a substantial  background 
of  knowledge  pertaining  to  the  implementation  and  operation  of  the  Single 
Integrated  Development  Test  Cycle  (SIDTC)  policy. 

1 . 2 Scope: 

This  study  paper  examines  actions  leading  to  initiation  of  the  new 
policy,  its  effect  on  roles  and  responsibilities  of  key  participants,  and 
relates  some  experiences  of  these  key  participants  in  the  operation  of 
programs  under  SIDTC. 

1 . 3 Specific  Goals: 

The  SIDTC  is  a newly  implemented  Army  test  policy  which  significantly 
redefines  the  conduct  of  Development  Test  and  Evaluation  (DT&E)  of  Army 
materiel.  Information  on  the  SIDTC  has  not  appeared  in  widely  circulated 
media  such  as  periodicals.  Conseoucntly , there  appears  to  be  a lack  of 
knowledge  of  tfr  new  policy  in  some  areas  of  the  Army  test  community. 
Specific  goals  of  this  study  project  were  to  acquire  information  to  under- 
stand and  relate  the  rationale  for  implementation  of  the  SIDTC  policy,  to 
understand  and  relate  changes  in  roles  and  responsibilities  of  major  parti- 
cipants, and  to  examine  and  describe  the  effect  to  de‘e  of  tne  SIDTC  policy 
on  manage:  r;t  and  conduct  of  r,"T  r tr  y ma  • ariol . 
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1.4  Limitations: 


The  SIDTC  policy  is  a very  broad  subject  with  many  ramifications.  The 
many  players,  roles,  and  responsibilities  involved  can  not  be  treated  in- 
dividually and  in  detail  within  the  constraints  of  this  study  project.  This 
paper  concentrates  on  major  aspects  of  the  development  test  structure. 
Operational  test  is  only  delt  with  to  show  how  it  relates  to  the  SIDTC. 

There  are  numerous  players  involved  in  the  operation  of  a program  under 
SIDTC.  However,  this  report  is  limited  to  those  participants  considered  to 
have  key  roles  on  the  development  side,  i.e.,  materiel  developer-PM, 
development  tester  (Tc.COM),  development  evaluator  (AMSAA),  and  contractor. 
The  SIDTC  policy  applies  to  major,  designated  non-major,  and  other  non-major 
programs.  In  this  report  structures  and  procedures  refer  to  the  major  and 
designated  non-major  program  orocess  unless  stated  otherwise. 

1 . 5 Method: 

This  report  is  eased  on  information  available  at  the  time  of  its  pre- 
paration. Doth  time  and  resources  were  limited.  Books  and  periodicals 
were  not  useful  sources  of  information  for  this  particular  subject.  Sources 
of  input  consisted  of  briefing  materials,  internal  documents,  correspondence 
between  Army  commands  and  activities,  Army  regulations,  and  informal  in- 
terviews and  phone  conversations  with  CARCOM  and  TECOM  personnel  who  pro- 
vided authoritative  information  on  the  subject. 
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BACKGROUND 

2 . 1 New  Concept  in  Army  Testing : 

In  January  1975  AMC,  now  Development  and  Readiness  Command  (DARCOM), 

12* 

initiated  a "new  direction"  in  Army  test  philosophy.  The  new  concept, 
termed  Single  Integrated  Development  Test  Cycle  (SIDTC),  through  imple- 
mentation letter  dated  21  January  1975  instructed  that  all  new  programs 
would  implement  the  policy  at  program  inception;  current  contracts  and 
on-going  programs  would  be  restructured  to  implement  the  new  integrated 
test  concept  to  the  maximum  extent  possible  within  funding  and  scheduling 
constraints . J 

The  new  direction  in  test  policy  was  designed  to  reduce  costs  and 
time  through  elimination  of  duplicate  testing  which  previously  had  been 
an  undesirable  characteristic  of  Army  developmental  testing  and  to  some 
extent  operational  testing  as  well.  To  provide  some  insight  into  the 
rationale  and  need  for  this  new  policy  some  of  the  history  leading  up  to 
it  will  be  related. 


‘Reference  sources  ure  identified  by  superscript  throughout  this  report 
and  are  keyed  numerically  to  Bibliography  listings. 
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2 . 2 Rationale  Leading  to  Change: 

In  1972,  as  a result  of  findings  by  a Department  of  Defense  Blue 

Ribbon  panel,  a requirement  for  operational  testing  to  be  accomplished 

by  an  agency  other  than  the  materiel  developer  was  made  mandatory  by  DA. 

This  led  to  formation  of  the  Operational  Test  and  Evaluation  Agency  (OTEA) 

and  its  sharing  of  the  Operational  Testing  (OT)  responsibility  with  TRADOC. 

OTEA  assumed  OT  responsibility  for  major  and  selected  non-major  systems 

12 

and  TRADOC  assumed  OT  responsibil ity  for  all  other  systems.  The  result 
was  four  separate  testers:  contractor,  developer,  TECOM,  and  the  operational 

tester  each  functioning  independently  of  the  others  and  conducting  tests  in 
a sequential  manner.  Overlapping  requirements  and  duplication  of  testing 
was  common  under  these  conditions. 

In  1974,  the  findings  of  other  investigative  groups  contributed  to 
another  significant  change  in  Army  test  philosophy.  Their  recommendations, 
basically  and  in  brief,  were  as  follow: 

a.  Army  Materiel  Acquisition  Review  Council  (AMARC) : 

- Transfer  TECOM  test  boards  to  TRADOC  to  conduct 
Operational  Tests  (OT)  and  Force  Development  Test  and 
Experimentation  (EDTSE). 

- Designate  Army  Materiel  Systems  Analysis  Agency 
(AMSAA)  as  independent  evaluator  of  Development  Tests 

1 9 

(DT)  and  developer  of  the  test  design.  *" 
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b . Point  AMC/Industry  Atlanta  Conference: 

- Reduce  excessive  testing  by  the  Government. 

- Reduce  duplication  of  contractor  testing  during  DT 

12 

by  the  Government. 

c . Research  Development  Acceptance  Test  (RDAT)  Study: 

- Establish  policy  of  single  integrated  contractor, 
developer,  AMSAA,  and  TECOM  test. 

- Structure  contracts  to  assure  contractor  demonstration 

of  essential  contract  compliance  to  obtain  confidence  for 
success  in  any  government  testing. 

- Eliminate  term  RDAT.  Replace  with  Prototype  Qualification 
Test  (PQT) . 

- Annotate  DD  Form  250  to  indicate  hardware  is  accepted 
for  test  purposes  only,  not  contract  compliance. 

- Require  formal  review  for  traceability  of  required  ROC,  DP, 

20 

and  System  specification  prior  to  proposal  solicitation. 

d.  Revised  AR  1000-1,  Basic  Policies  for  Systems  Acquisition  by  the 
Department  of  the  Army,  5 November  1974. 

- Establish  Single  Integrated  Development  Test  Cycle  (SIDTC) 

Pol  icy: 

"During  the  development  phase  of  the  acquisition  cycle 
the  larger,  more  sophisticated  sysie  s will  hr  subject  to  only 
two  test  cycles.  The  contractor  and  materiel  developer  develop- 
mental testing  should  be  integrated  into  one  cycle  with  operational 
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testing  constituting  the  other  test  cycle.  For  the  smaller,  less 
sophisticated  systems,  sufficient  prototype  hardware  can  on  occasion 
be  provided  that  would  permit  totally  independent  materiel  developer 
testing.  The  policy  of  coordinating  test  planning  with  the  ob-  -j 
jective  of  minimizing  the  number  of  tests  applies  to  all  systems." 

In  essence,  these  findings  could  have  been  interpreted  thusly: 

- An  integrated  test  philosophy  to  eliminate  duplicatory  materiel 

testing  would  greatly  facilitate  a more  effective  and  efficient 

policy  in  the  conduct  of  army  testing. 

The  Single  Integrated  Development  Test  Cycle  (SIDTC)  was  promulgated 
to  accomplish  this  objective. 
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SECTION  III 


OBSERVATIONS  AND  DISCUSSION 

3.1  Responsibilities  Under  Single  Integrated  Test  Cycle  (SIDTC): 

Execution  of  SIDTC  required  acceptance  of  new  roles  and  responsibilities 
by  the  major  players  involved.  Major  players,  in  this  instance,  refers  to 
the  Developer  (MSC/PM) , AMSAA,  and  TECOM.  However,  the  expanded  role  of 
the  contractor  and  inclusion  of  the  OT  interface  should  not  be  diminished. 

The  following  responsibilities  associated  with  SIDTC  were  delineated  in 
DARCOM's  Letter  of  Implementation. 

RES  PON  SI  RIL.ITI.ES_  OF  THE  DEVELOPER  (MSC/PM) 

AMSAA  AND  TfCOM 

1.  Responsibilities  of  the  Developer  (MSC/PM): 

a.  Preparation  of  the  Coordinated  test  Program  (CTP)  in  coordinatio; 
with  the  Combat  Developer,  Operational  Tester,  Logistician,  AMSAA 
TI.COM  and  with  appropriate  contractor  participation. 

b.  Management  of  the  Development  Test  portion  of  the  Coordinated 
Test  Program  ( CTP ) . ^ 

2.  Responsibilities  of  AMSAA: 

a.  Preparation  of  the  Independent  Evaluation  Plan  for  DT  I,  DT  II, 
and  DT  III  for  major,  designated  non-major  and  other  selected 
systc  r. ; a 1 so  to  overview  the  o'. Her  syst-  s on  a samp  1 inn  fasi 
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This  plan  details  actions  for  acquiring  sound  test  data  responsive 
to  the  decision  process  and  provides  the  basis  for  formulation  of 
the  overall  test  design.  The  plan  is  coordinated  with  the  Developer 
(MSC/PM)  and  TECOM  and  is  provided  as  input  to  the  Coordinated 
Test  Program  (CTP). 

b.  Preparation  of  the  overall  test  design  for  DT  I,  DT  II,  and 
DT  III  for  major,  designated  non-major  systems  and  other  selected 
systems.  The  test  design  serves  as  the  basis  for  determining  those 
tests  that  will  be  performed  by  the  contractor,  .VIC  major  subordinate 
commands,  project  managers  and  TECOM  which  will  be  used  for  the 
overall  system  evaluation.  The  test  design  is  coordinated  with  the 
Developer  and  TECOM,  and  provided  as  input  to  the  Coordinated  Test 
Program  (CTP). 

c.  Preparation  of  the  Independent  Overall  Evaluation  of  Major  and 

designated  non-major  systems  and  other  selected  systems  for  pre- 
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sentation  to  the  Developer  and  CG  AMC  [DARCQM.,. 

3.  Responsibilities  of  TECOM: 

a.  For  systems  which  AMSAA  is  not  the  evaluator: 

(1)  Preparation  of  the  Independent  Evaluation  Plan  fot  DT  I, 

DT  H and  DT  III.  Tins  plan  details  actions  for  acquirer-  sour,! 
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test  data  responsive  to  the  decision  process  and  provides  the  basis 
for  formulation  of  the  overall  test  design.  The  plan  is  developed 
in  coordination  with  the  ‘ISC's  materiel  proponent  and  is  provided 
as  input  to  the  Coordinated  Test  Program. 

(2)  Preparation  of  the  overall  test  design.  The  test  design 
serves  as  a basis  for  determining  those  tests  that  will  be  performed 
by  the  contractor,  AMC  [DARCOM]  major  subordinate  commands,  project 
managers  and  TECOM,  which  will  be  used  for  the  overall  evaluation. 
The  test  design  is  coordinated  with  the  MSC's  materiel  proponent  and 
provided  as  input  to  the  Coordinated  Test  Program. 

(3)  Preparation  of  the  Independent  Overall  Evaluation  in  co- 
ordination with  the  MSC  "Red  Team"  for  presentation  to  the  Developer 
and  CG  AMC  [ DARCOM J . 

b.  For  all  systems,  TECOM  will  plan,  conduct  and  prepare  analysis 
of  government  validation  tests,  when  required.  (Advanced  Develop- 
ment Verification  Test  - Government , Prototype  Qualification  Tert  - 
Government,  arid  Production  Validation  Test  - Government).  Tin  MSC's 
materiel  proponent  a no'  /•  MSAA  will  also  conduct  their  own  inde- 
pendent analyses  of  these  test  results,  as  required.  TECOM  will 
provide  lest  reports  and  analysis  of  test  results  to  the  MSC's 

S 

materiel  proponent  and  ■MSA/  for  systems  evaluation. 
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3.2  T est  Integration  Working  Group  (TIWG): 


Early  coordination  of  test  requirements  for  DT  and  OT  became  an  integral 
part  of  successful  execution  of  the  integrated  test  cycle.  To  facilitate  the 
integration  of  test  requirements  and  to  speed  the  coordination  process,  the 
materiel  developer  is  required  to  establish  and  chair  a test  Integration 
Working  Group  (TIWG)  for  each  major  and  selected  non-major  program.  The 
qrcup  is  formally  chartered  and  consists  of  reqpresentati ves  having  authority 
to  act  for  their  respective  commands  or  activities.  However,  they  do  not 
hove  the  authority  to  approve  the  Development  Plan  (DP),  alter  system 
specifications,  or  address  agenda  items  not  pertinent  to  or  coordinated 
with  principal  members.  For  systems  other  than  major  and  selected  non-major 
the  decision  to  form  or  not  to  form  a TIWG  is  made  by  a joint  materiel 
developer  - combat  developer  working  group.  TIWG  membership  includes  as 
a minimum  representatives  of  the  Combat  Developer,  Logistician,  Operational 
Tester,  AHSAA,  TECOM  and  where  appropriate,  the  contractor.  Primary  purpose 
of  the  TIWG  is  to  assist  the  developer  in  the  preparation  of  the  Coordinated 
Test  Program  (CTP),  monitor  test  program  progress,  and  update  the  CTP  when 

• i 5 

required . 

3.3  Coordinated  Test  Program  (CTP): 

The  CTP  is  the  key  management  document  for  assuring  that  all  appropriate 
testing  accomplished  by  the  contractor,  proponent  materiel  developer,  TECGM, 
and  the  operational  tester  is  properly  planned,  coordinated,  conducted, 
analyzed,  and  reported.  The  materiel  developer  (I'SC/FW)  is  responsible 
for  preparation,  coordination,  distribution,  and  updating  the  C1P.  Co- 
ord  i no  * i on  md  ccn ' amrices  involve  all  prir'rv  -embers  o*  the  TIWG.  ihir 
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document  identifies  required  testing,  test  personnel,  organizations  involved, 

materiel,  facilities,  troop  support,  logistic  support,  funds,  critical 

issues,  and  test  criteria  for  implementing  the  test  program.  It  provides 

assurance  that  DT  and  OT  design  plans  will  be  coordinated  so  that  each 

test  cycle  requires  minimum  resources  and  yields  maximum  data  to  satisfy 

the  common  need  of  the  materiel  developer  and  the  operational  tester.  The 

CTP  is  used  to  plan,  coordinate  and  integrate  the  scheduling  of  all  DT  and 

OT  and  any  other  testing  required  for  an  item  or  system.  As  such,  it 

4 

provides  a complete  testing  perspective  for  the  program. 

3.4  Formulation  and  Preparation  of  the  CTP: 

The  Coordinated  Test  Program  is  not  a new  document  invented  just  for 
SIDTC,  it  was  a part  of  previous  Army  test  planning  concepts.  However, 
the  CTP  was  reformated  to  complement  the  new  integrated  test  policy. 

Actually,  the  new  CTP  is  net  a single  document  covering  all  phases  of  the 
acquisition  cycle  as  was  the  case  in  the  past.  In  SIDTC  three  separate 
documents  are  written.  Each  document,  addresses  the  appropriate  testing  to 
be  accomplished  during  the  particular  phase  of  the  materiel  acquisition 
process  for  which  it  applies.  There  is  the  CTP- 1 for  Validation  phase 
testing,  a CTP-II  for  Full  Scale  Development  phase  testing,  and  a CTP - 111 
for  the  Production  and  Deployment  phase.  It  may  not  be  excessive  to  de- 
scribe the  CTP  and  Tli-M  as  being  central  to  the  SIDTC  operations.  The  coordi- 
nation of  effort  evolving  out  of  TIT'S  and  CTP  activities  has  been  described  as 
one  of  the  keystones  of  the  SIDTC.  (see  paragraph  3.6)  It  is  through  the  Tl.-.'T 
and  CTP  formulation  and  preparation  that  a major  ini  rface  with  the  opei  iti  nal 
test  proponent  occurs.  1 ,ures  2 ana  3 vow  a tjoical  formulation  .;rvl  . r-  - 
puration  flow  pr._jiS  for  :■  vdop  • ' of  t > C~  . 
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All  coordination  and  integration  are  accomplished  by  making  maximum 

1 3 

utilization  of  the  TIWG.  Figures  4-9  show  a more  detailed  breakdown  of 
the  process.  All  figures  are  greatly  simplified  and  are  intended  to  in- 
dicate a general  process  which  occurs  for  each  phase  of  the  acquisition 
cycle.  Complete  information  on  the  CTP  and  TIWG  are  too  extensive  to 
include  in  this  study  paper.  Details  are  contained  in  DA  Pamphlet  70-21, 

The  coordinated  Test  Program. 

3.5  I J ey i_  Test  _D efi ni  ti ons : 

The  SIDTC  concept  introduced  new  test  terminology  into  the  test 
community.  Initially  SIDTC  terminology  did  not  receive  wide  acceptance  and 
usage  in  day-to-day  activities.  Many  involved  in  testing  preferred  to 
continue  using  terms  associated  with  the  previous  test  concept  such  as: 

4 

RDAT,  ET,  ST,  and  IPT.  Figure  10  illustrates  the  relationship  of  new 
test  terms  to  the  time  phased  acquisition  cycle  and  decision  points.  The 
list  which  follows  defines  major  tests  normally  conducted  during  DT  I,  DT  II, 
or  DT  III  of  a development  effort. 

EXPLANAT IOJ,'  OF  terms 

Advanced  Devolo""ont  Verification  Test-Contractor  (ADVTC-C).  A test 
conducted  by  a contractor  during  the  Validation  Phase  on  components,  sub- 
systems and/or  systems  to  demonstrate  that  contract  materiel  requirements 
have  essentially  been  at  and  provide  a significant  degree  of  confidence 
that  the  materiel  will  successfully  undergo  and  complete  essential  sub- 
sequent govern  ni.  t.es:ing.  For  r.on-cc  peting  systems  tnis  test  may 
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Figure 


constitute  the  total  verification  testing  required. 


Advanced  Development  Ve ri fic ation  Test-Government  (ADVT -Gj . A test 
conducted  by  TECOM  during  the  Validation  Phase,  only  when  essential,  to 
independently  validate  contractor  test  results  and  execute  those  tests 
beyond  the  capabilities  of  the  contractor.  This  test  is  the  final  veri- 
fication test  prior  to  the  decision  to  enter  the  Full-Scale  Development  Phase. 

Engineering  Design  Test-Contractor  (EDT-C).  A test  conducted  by  a 
contractor  during  the  Validation  or  Full-Scale  Development  phase  to  deter- 
mine ach ievabil i ty  of  critical  system  technical  characteristics , provide 
data  for  refining  and  ruggedizing  hardware  configurations,  elminiate 
technical  design  risks  or  determine  manageability  of  risks,  and  provide 
for  evolution  „nd  verification  of  design  changes. 

Engineering  D sign  Test-Government  (EDT-G) . A lest  conducted  by 
developer  during  the  Validation  or  Full-Scale  Development  Phase  to  deter  ne 
achievabil ity  of  critical  system  technical  characteristics,  provide  data 
for  refining  and  ruggedizing  hardware  configurations,  eliminate  technical 
design  risks  or  determine  manageability  of  risks,  and  provide  for  evolu- 
tion and  verification  of  design  changes. 

Prototype  Qualification  Tost-Contractor  (POT-C).  A Lest  conducted  by 
a contractor  during  the  Full-Scale  Development  Phase  on  components,  sub- 
systems and/or  systems  to  demonstrate  that  contract  materiel  requirements 
have  essentially  been  met  and  provide  a significant  degree  of  confidence 
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that  the  materiel  will  successfully  undergo  and  complete  essential  sub- 
sequent government  testing.  This  test  incorporates  the  purpose  of  the 
former  Research,  Development  and  Acceptance  Test  (RDAT). 

Prototype  Qualification  Test-Goverir  ent  (PQT-G).  A test  conducted  by 
TECOM  during  the  Full-Scale  Development  Phase,  only  when  essential,  to 
independently  validate  contractor  test  results  and  execute  those  tests 
beyond  the  capabilities  of  the  contractor.  This  test  is  the  final  qualifi- 
cation test  prior  to  a production  decision. 

Production  Validation  Test-Ccntractor  (PVT-C).  A test  conducted  by  a 
contractor  during  initial  production  to  demonstrate  that  contract  materiel 
requirements  have  been  met,  prior  to  subsequent  government  testing. 

Pi  ' ficn  Validation  Test-Goverm  ent  (PVT-G).  A test  conducted  by 
TECOM  during  initial  production  to  independently  validate  contractor  test 
results  (only  when  essential)  and  execute  those  tests  beyond  the  capabilities 
of  the  contractor.  This  test  is  the  final  validation  test  prior  to  pro- 
duction and  deployment.^ 


3.6  Key  Elements  of  an  Operative  SIDTC: 

Five  key  areas  have  been  cited  as  major  elements  for  the  successful 
functioning  of  the  SIDTC: 

- Coordination  of  effort 

- Utilization  of  contractor  test  data 

- Reduction  and  integ-'atior  ol  tests 

- Increased  early  soldier  participation 

- F.'  : 'hasized  in  !••;  on  ’ .e  f . alu  •*  vs  t eg 


1.  Coordination  of  Effort: 


More  than  ever  before  the  materiel  developer  (PM)  is  the  focal 
point  for  program  management  responsibi 1 ity . Although  the  developer, 
contractor,  development  and  operational  testers,  development  and  operational 
evaluators,  trainer,  and  logistician  are  all  intimately  involved  in  the  co- 
ordination process,  recognition  of  PM  leadership  is  very  important  to  the 
success  of  SIDTC  efforts.  The  materiel  developer  must  coordinate  the 
efforts  of  all  participants  and  integrate  valid  requirements  into  cost 
effective  development.  The  primary  vehicle  utilized  for  coordination  is 
the  TIWG.  TIWGs  should  be  composed  of  cooperative  individuals  with  the 
authority  to  make  concessions  with  respect  to  the  parochial  interests  of 
the  participants.  As  was  pointed  out  earlier,  an  important  effort  of  the 
project  manager  and  TIWG  is  to  develop  and  maintain  the  CTP  to  insure  that 
maximum  integration  has  been  effected  and  that  the  test  design  is  appropriate 
and  cost  effective.^ 

2.  Utilization  of  Contractor  Test  Data: 

Maximum  utilization  of  validated  contractor  test  data  is  another  major 
element  in  the  SIDTC  concept.  In  the  past.  Army  testers  made  little  use  of 
contractor  test  data,  they  were  more  intent  on  generating  their  own  re- 
sources independent  from  the  contractor.  'low,  extensive  monitoring  of 
contractor  testing  and  utilization  or  validated  contractor  test  results 
to  satisfy  Government  test  require:  ents  arc  essentials  of  effective  SIDTC. 
Government  test  vs  contractor  test  require  nits  must  he  included  in  the 
RFP,  L01  and  clearly  delineated  in  the  contract.  The  contractor's  share 


of  development  testing  has  increased  and  test  resources  available  for 
contractor  use  has  also  increased.  Contractors  should  be  encouraged  to 
utilize  Government  test  facilities  a the  proving  grounds  and  ranges  on  a 
reimbursement  for  direct  cost  basis,  l.nen  feasible,  contractors  can  conduct 
contractor  tests  at  Government  test  sites  with  contractor  personnel  or 
Government  personnel  can  perform  the  tests  strictly  on  a customer  basis.11 

3 . Reduct  ion  and  Integrati on  of  Tes ts : 

Reduced  test  schedules  and  duplication  of  the  development  effort 
are  basic  objectives  of  SIDTC.  Integration  of  Government  and  contractoi 
testing  is  the  primary  r cans  of  accomplishing  the  reduction.  Ot  .er  means 
may  also  be  considered.  Production  decision  point  may  be  achieved  earlier 
by  integrating  test  phases.  For  example,  overlap  between  OT  I and  DT  II 
or  between  OT  III  arid  OT  III  may  be  viable  possibilities.  It  may  also  be 
possible  to  reduce  the  scope  of  DT/OT  I and  III.  Combined  DT/OT  I and 
DT/OT  III  could  also  be  accomplished  in  some  cases.'1 

4 . Early  Sol d_i er_  Par t i ci pati on : 

Early  user  or  troop  participation  is  another  major  element  in 
SIDTC.  Solutions  to  user  equipment  problems,  if  surfaced  early  in  the 
development  cycle  insures  compatibility  of  critical  man-machine  relationships 
and  acceptability  of  design  strategy.  This  action  alone  can  result  in 
elimination  of  costly  and  time  consuming  design  changes  which  can  practically 
stymie  a development  program  if  they  arc  surfaced  downstream.11 


?b 


5 . Independence  of  Evaluation  and  Test: 

The  new  emphasis  on  independent  evaluation  and  independent  test 
leads  to  more  efficient  test  design  and  program  structures.  Increased 
technical  planning  requires  more  intensive  early  management  and  early 
commitment  of  increased  resources.  All  of  this  should  have  a positive 
result  on  the  overall  development  program  and  contribute  to  credibility  of  the 
SIDTC  process.^1 

3.7  Program  Successes  with  SIDTC: 

At  the  time  of  writing  of  this  report  (October  1976)  the  SIDTC  policy 
has  been  in  effect  approximately  twenty-one  months  based  on  the  Implementation 
Letter  of  21  Januay  1975,  as  the  reference  point.  Quantified  data  as  to 
success  or  failure  of  the  SIDTC  relative  to  specific  programs  utilizing  it 
was  difficult  to  obtain  by  the  writer  in  the  limited  time  permitted  to  research 
this  report.  The  indication  is  that  little  quantified  data  has  been  cor- 
related and  published  at  this  time.  However,  there  is  evidence  in  some 
programs  of  positive  results  which  can  he  attributed  to  utilization  of 
SIDTC  or  elements  of  the  new  concept,  notable  examples  of  SIDTC  benefits 
to  five  major  programs  have  been  reported. 

1 • /"  : 

Tin's  program  involved  DA  effort  to  reduce  the  number  of  HLLLFIPL 
Anti-Tank  Missiles  in  a test  program  associated  with  the  development  of 
the  Advanced  Attack  Helicopters.  A special  task  force  reviewed  the  programs 
and  devised  an  integrated,  coordinated  test  plan  for  all  of  the  systems. 


It 


Through  concurrency  of  test  phases,  earlier  operational  testing,  and 
testing  the  HELLFIRE  on  a proven  airframe,  a shorter  test  cycle  resulted 
from  earlier  production  decisions.  The  result  was  a reduction  of  90  HELLFIRE 
missiles,  a total  savings  to  all  programs  of  $138  million,  and  earlier  initial 
operational  capabilities  of  approximately  one  year.  Drawbacks  identified 
were  additional  early-on  costs  and  increased  management  risks  due  to  the 
complexity  of  the  integration  of  the  systems . 

2.  XM1 : 

The  XM1  tank,  designed  to  provide  a significant  improvement  in 

ground  combat  power  has  integrated  contractor  and  Government  testing  and 

utilized  contractor  data  to  reduce  the  scope  of  Government  validation  testing. 

Up  to  twenty-six  percent  of  contractor  firing  data  was  utilized  to  satisfy 

11 

a portion  of  Government  testing.  Other  contractor  data  were  used,  however, 
quantified  benefits  from  this  data  were  not  reported. 

3.  UTTAS: 

Tiie  Utility  Tactical  Transport  Aircraft  System  is  another  program 
in  which  SIDTC  concepts  were  applied.  It  is  reported  that  this  program  lias 
been  reduced  to  an  absolute  minimum  in  length.  ^ 

4.  MICV: 

The  Mechanical  Infantry  Combat  Vehicle  while  undergoing  contractor 
tests  at  Government  test  sites  (Aberdeen  and  Yuma  Proving  Grounds)  utilized 
soldiers  to  test,  maintain,  and  evaluate  the  vehicle. This  program 
furnishes  examples  of  two  elements  in  the  SIDTC  process:  (a)  contractor 

testing  at  Government  test  facilities,  and  (b)  soldier  input  and  participation 
early  in  the  life  cycle. 
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5.  SAM-D  (PATRIOT): 


In  this  program  the  coordinated  efforts  of  the  major  participants 
resulted  in  development  of  a firing  matrix  to  satisfy  both  industry  and 
Government  test  data  requirements.  The  approach  was  expected  to  save 
dollars  and  through  improved  test  design  permit  a better  evaluation  to  be 
available  early  in  the  program  to  support  key  production  decisions.^ 

3 . 8 Other  Observations: 

1 . Developer PM  Related 

Initially  it  was  believed  that  an  adversary  realationship  would 
develop  between  the  evaluator  and  developer  - P’1  as  a result  of  SIDTC 
emphasis  on  independent  evaluation  of  validated  test  data  as  opposed  to 
independent  testing  which  in  the  past  was  suspected  of  being  too  close  to 
the  developer.  Some  PM's  appeared  to  feel  that  addition  of  the  independent 
evaluator  would  cause  unnecessary  cost  increases  and  delays  in  their  prograi 
This  apprehension  seems  to  have  been  largely  dispelled  as  front-end  loading 

of  programs  has  tended  to  reduce  surprises,  reduce  program  bias  in  dcti  ion 

1 8 

making,  and  help  to  increase  credibility  of  test  results.  All  of  these 
are  positive  benefits  to  the  PM. 

2.  TECQM  Related: 

a.  There  is  evidence  of  reduction  in  duplication  of  testing  by 
Government  testers.  However,  increases  in  successful  Govern  ent  test’in 


?o 


after  contractor  tests  have  been  performed  cannot  be  conclusively  stated 
at  this  time.  It  is  believed  that  these  results  will  become  more  evident 


in  another  year  or  two. 

b.  Increased  monitoring  by  TECOM  representati ves  at  contractor- 

facilities  have  in  some  cases  caused  stresses  in  manning.  Insufficient 

personnel  to  accommodate  the  increased  and  extended  TDY  duties  on  a timely 

basis  could  cause  perturbations  in  monitor  activities  or  manning  of  test 

programs  being  conducted  simultaneously  at  the  proving  grounds.  Where 

contractor  tests  are  used  to  satisfy  TECOM  testing  contractor  test  plans 

could  be  improved  to  include  sifficient  detail  to  obviate  1 ECOM  rewriting 
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of  the  plans  to  meet  Government  test  standards  and  regulations.  These 
rewrites  consume  additional  manpower  and  time.  Lack  of  coordination  of 
contractor  test  plans  was  pointed  to  as  a lingering  problem. 

c.  Environmental  testing  by  the  contractor  has  in  some  instances 
been  an  issue.  Instances  were  cited  in  which  contractor  environmental 
chambers  were  inadequate  or  outmoded.  In  these  cases  doubts  as  to  the 
contractors  compliance  with  Military  Standard  and  Government  test  regulations 
were  very  real,  raising  serious  questions  as  to  the  validity  of  the  test 
data.  In  some  cases  the  contractor  has  been  reluctant  to  take  equipment 

to  TECOM  facilities  fo*-  conduct  of  these  tests.  In  general,  there  seems  to 

be  a reluctance  or,  the  part  of  the  contractors  to  utilize  TECOM  test  facilities 

for  conducting  contractor  tests.  However,  requests  for  use  of  TECOM  test 

1 S 

facilities  are  reported  co  be  ;ncreasirq. 
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d.  Minor  difficulties  hove  been  experienced  in  getting  TECOM 
tests  delineated  in  the  RFP.  This  is  hardly  a problem  where  major  systems 
are  concerned,  but  is  more  evident  with  smaller,  non-major  systems.  The 
difficulty,  where  experienced,  apparently  stemmed  from  the  developer's 
failure  to  involve  TECOM  early  enough  in  the  program.  Effort  in  the  form 

of  additional  guidance  from  DARCOM  has  been  made  to  improve  this  situation.6 

e.  It  was  hypothesized  that  increased  costs  associated  with  TIl.'G 
attendance  and  extended  TOY  required  for  close  monitoring  of  tests  at  the 
contractor's  plant  would  negate  possible  dollar  savings  from  combining  test 
requirements.  At  this  point  evidence  indicates  that  savings  due  to  inte- 
gration of  tests  are  not  off-set  by  increased  TINS  and  TOY  costs.  Initial 
test  cost  considerations  are  made  when  the  Independent  Evaluation  Plan  (IEPl, 
and  CTP  are  developed  and  ample  opportunity  exists  to  make  comparisons  and 
determine  which  course  is  most  feasible  and  economical.^ 

f.  A more  serious  problem  associated  with  the  new  test  concept 

is  the  potential  for  proliferation  of  developmental  testing  outside  of  TECOM. 
The  danger  exists  that  FMs  and  commanders  will  tend  to  utilize  materiel 
developer's  test  facilities  in  lieu  of  TECOM  test  sites.  A notable 
example  of  this  type  of  problem  was  the  PM-EAMECE  proposal  to  conduct  the 
Government  portion  of  DT  II  (PQT-G),  i.e..  Prototype  Qualification  Test- 
Government,  at  the  MERADCOM  location  rather  than  at  a TECJM  facility.1' 
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Although  the  PM-FAMECE  decision  was  upheld  as  being  more  feasible  in  this 
case,  the  problem  was  considered  serious  enough  to  prompt  a caution  from 
DARCOM  against  potential  proliferation."’^’^ 

3 . Eva luat or  Related: 

a.  Questions  raised  at  the  out-set  of  implementation  as  to  AMSAA's 
credibility  as  a qualified  independent  evaluator'^  for  some  programs  have 

in  most  instances  been  dispelled  and  credibility  is  increasing  as  AMSAA 
becomes  more  attuned  to  pragmatical  aspects  of  the  developer's  testing 

needs.  ^ 

b.  AMSAA  now  presents  at:  independent  evaluation  to  the  developer 
and  Commanding  General,  DARCOM.  Each  program  is  treated  on  a case  by  case 
basis.  Headquarters,  DARCOM  determines  if  the  independent  evaluation  will 
be  folded  into  the  DA  IF R/ASARC/DSARC  or  presented  separately. 

c.  Stresses  due  to  increased  manpower  requirements  appear  to  be 

a significant  factor  for  evaluators  for  both  major  and  non-rnajor  evaluations. 
The  number  of  programs  initially  to  be  ohased-in  by  AMSAA  for  evaluation 
was  subsequently  revised  downward  and  some  of  the  programs  were  assigned 
to  TECOM-MSC/Red  team  for  evaluation.  Another  indication  of  the  increased 
manning  requirement  may  be  derived  from  the  ARMCOM  report  on  scope  and  man- 
power requirements  cf  the  ARMCOM  System  Analysis  Directorate  in  support 


of  the  Test  Design  and  Evaluation  effort  as  part  of  the  SIDTC  policy  in 
evaluation  of  development  testing  of  systems  not  evaluated  by  AMSAA.  ARMCOM 


Systems  Analysis  reported  the  need  for  25  additional  full  professional 
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personnel  to  fulfill  mission  responsibilities  resulting  from  SIDTC. 

4 . Contract or _R  el ated : 

a.  Use  of  DD  Form  250  to  remove  equipment  from  the  contractor's 
plant  for  test  only  prior  to  formal  acceptance  by  the  Government  has  not 
been  a problem  with  the  contractor.  It  is  recognized  that  the  purpose  of 
the  DD  250  in  this  respect  is  to  show  who  has  responsibility  for  safeguarding 
and  possession  of  the  equipment.  In  such  cases  the  Government  is  the  liable 
par  ty . ^ ^ 

b.  Contractor  participation  in  test  planning  and  use  of  Government 
facilities  for  contractor  tests  iias,  in  general,  not  been  a problem.  These 
arrangements  are  determined  at  the  time  of  RFP  preparation.  The  contractor 
must  then  be  aware  of  each  test,  where  it  will  be  performed,  the  associated 
costs,  and  who  is  going  to  pay  whom  for  what.  Actually,  this  type  of 
contractor  involvement  has  proven  to  be  an  assist  in  terms  of  definitiver.ess 


— 


of  testing  arrangements . 
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SECTION  IV 
SUMMARY 

4 . 1 I implementation : 

In  the  opinion  of  a DARCO‘1  SIDTC  authority,  the  SIDTC  policy  is 

certainly  fully  implemented  at  this  time.  It  was  estimated  that  the  policy 
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could  have  been  considered  fully  implemented  by  January  1976.  ' DA 
components  and  activities  appeared  to  react  well  to  the  implementation 
instructions.  Implementation  by  new  programs  was  easier  and  faster  than 
for  those  programs  which  were  on-going.  On-going  programs  which  were  in 
various  stages  of  development  caused  some  delay  in  full  implementation. 
However,  these  delays  were  anticipated. 

4 . 2 Impact  of  SIDTC: 

1.  The  SIDTC  concept  appears  to  have  been  highly  beneficial  to  new 
programs  which  have  utilized  it  from  inception  or  from  early  stages  of  the 
development.  For  on-going  programs  benefits  have  been  less  evident,  possibly 
because  the  advantages  that  accrue  to  intensive  early-on  management  could 
not  be  realized  or  problems  resulting  from  lad.  of  it  had  already  started. 

Tne  policy  has  had  the  desired  effect  of  reducing  duplicatory  testing, 
but  the  problem  of  proliferation  o4'  materiel  developer  testing  outside  or 
TT COM  iias  surfaced  and  could  cause  future  problems. 
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2.  A major  benefit  resulting  from  SIDTC  has  been  the  improved  co  uni- 
cations throughout  the  test  community  especially  in  terms  of  communication 
with  the  operational  test  proponents.  Although  integration  of  operational 
testing  was  not  formally  included  in  initial  implementation  plans,  the 

policy  was  used  as  a jumping-off  point  to  expand  interaction  with  operational 
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testers  and  OTEA.  This  expanded  interface  is  evidenced  by  the  increased 
operational  participation  and  coordination  in  all  test  planning  particularly 
as  it  applies  to  TIUG  activities  and  development  of  the  CTP. 

3.  There  is  not  sufficient  quantified  data  available  as  yet  to  fully 
evaluate  the  SIDTC,  particularly  in  terms  of  dollar  savings.  Such  an 
evaluation  will  become  feasible  as  additional  new  programs  fully  utilize  the 
new  concept  and  as  more  information  becomes  available  from  programs  currently 
utilizing  it  as  these  programs  move  downstream  in  the  development  cycle.  Al- 
though quantified  data  is  insufficient  to  reach  definite  conclusions  to  date, 
the  potential  for  savings  in  cost  and  time  are  evident.  Limited  information 
to  this  effect  has  been  reported.  Dollar  savings  and  reductions  in  the 
acquisition  life  cycle  are  expected  to  become  more  evident  as  participants 
become  more  accustomed  to  SIDTC  practices. 
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